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FACTOR ANALYSIS OF LAW SCHOOL GRADIS 
Robert F. Boldt ^ 

Abstract 

Grades of 116 law school students were obtained from transcripts >* 
along with LSAT, Writing Ability, and Backgroimd scores. The grades 
were factored ^ and an analysis of the goo.dhess of fit of one- through 
four-factor systems indicated that the system of grades was essentially 
one factor in nature, A plot of factor loadings for the various courses 
made for the two^ and three-factor systems indicated: that a few courses 
were somewhat different than the rest: biit the reason .was hot clearly 
identifiable. The two-factor system was decdmppsed into two uncorre- 
iated components^ one which correlated .96 with the single factor scores, 
and the other which correlated zero with the single factors scores . This 
latter component had a multiple correlation of .03 with LSAT, Writing 
Ability, and Background scores. These analyses produced no persuasive 
evidence for more than one factor. The results agree with those of a 
similar study on the business school context. 

Ihe residuals produced were used to study the possible existence 
of instructor bias. Two instructors were found with resriduals signifi- 
cantly different from zero. One was found to be a slightly easy grader, 
and the other was found to be a hard grader. 



FACTOR ANALYSIS OF LAW SCHOOL GRADES 
Robert F. Boldt 
Educational Testing Service 

Many educational researchers have concluded that the limits of 
validity have been reached due to the complex nature of the grade point 
criterion^ This presumed complexity arises in part from the diverse 
nature of the courses that may be taken by the given student^and^the 
corresponding abilities that may be required. For example, in business 
education one might expect a course on the international aspects of 
business to require different abiii^ies for successful performance than 
wdtild a course in co^uter operations of cS^^^ In fact ^ one 

might expect to find a group of courses which could reasonably be called 
the "verbal group" based both on their content and their relationship 
between grades in these courses and' scores oh a test of verbal ability* 
Similarly one might find "technical groups** which relate to quantitative 
ability. At least such were the expectations when a facto?%.analysis of 
graduate business school grades was conducted. At two graduate schools of 
business the grades of a^ graduating class were examined to determine the 
number of abilities regiuired to account for the academic performances. At 
both schools, only one factor, i.e., one ability, seemed to underlie the 
business school grades. This was in contrast to the strong presumption 
that some courses would be primarily dependent on verbal abilities and 
other courses would depend primarily on quantitative or technical skills. 
It had been thought at the outset of these studies that the courses could 
be grotiped so that courses in a single group would all require the same 
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balance of abilities. If more than one such group was found, then the 
relation between performance in courses within a group and test scores 
could lead to an identification of the nature of the predictable parts of 
the grade point criterion. If several abilities had been discovered, then 
the grade point average could be fractionated into relatively pure sub- 
criteria-and test development efforts initiated to improve prediction for 
those courses hot currently well predicted. With the single ability result, 
however, there was ho possibility of such fractionation, and, in fact, the 
value of the single jgrade point average as a criterion was supported. 

Within the context of legal education there is reason to suspect that 
perhaps the unidimensiohal character of; the factor structure of grades 
might not obtain.,. There has been a changing concern in recent years with 
the sociological relationship between law and society, and this concern, 
among others, is manifested in electives or in the second and third years. 
It is thought that possibly the traditional concerns reflected in the first 
year of law training may as a group require different abilities than the 
broader exposures occurring later on. To the extent that this is true, 
different legal courses may require different balances of common abilities 
for excellence of performance and the grade point criterion might be modi-- 
fied as was suggested in the business school context. If only a single 
factor is found, then one may conclude that though the^ subject matter 
learned is different in content, the abilities required for its mastery 
are similar or are unique for each course, i.e., the change in subject 
matter does not imply a change in the nature oX the predictors required. 

Finally, there is a concern with instructor differences in grading. 
It is reasonable to suspect that different instructors grade with different 



-3- 



reliability. Teaching approaches differ as »y testing practices, 
tor thl. reason, an ana..,ls of Instructor differences was aUo conducted 
in this study. 

MPthodologj ''''T Orientation 

.he se,uence of cents In conducting the study begins with an attest 
ro determine the structure of ability re,„lre«nts for the various courses. 
«.ls is achieved by associating certain para«ters with student., and certain 
„ther parameters with courses *d.by deter-nlng how well suitable co«,lna. 
.ions of these para.e..-s s«arl.e course grades. *or e:.a^le. when three 
: eblUtles ar. PO.tuUt.d. the p*r,«Ur. .ssa^lated .i.h -the Business Schoo 
cou.se ■■Organl.atlon and Leadership of Enterprises" (.oldt. 1970,; were found 
to be 

a, - .007 . a, - -0" .. a, < -.1«9 . b -2.627 
and the parameters associated with a particular student were 

z - 64.227 , ^2 - 1-096 • ^""^ H ' " 
The a -s, b -s, and . 's are coMblned to get an approxl«te grade as 

follows: 

^1^1 2 2 3 3 
. (.GOV) (64.227) * (-.041) (l'.096) * (-IW (-'W * 2-'" 

« 2.928 . 

Xhe student Whose parameters are given above actually obtained a grade of 
3 (C) in "organization and Leadership of Enterprises"; thus the three- 

^«»^.d his course performance quite well, leaving a 
ability system approximated his course v 

* i„ 072 n 000 - 2.928) on the grade point scale. It 
residual error of only .072 (i.uuu 



is these squared residuals ^ summed over all people and over all the courses 
they took, that the analysis minimizes. 

Notice that the scores associated with people, often referred to in 
the factor analy^:ic literature as factor scores, ara here called ability 
scores. This is considered reasonable in 'the sense that variations in 
z *s produce varying approximated performances in the same course. The 
number of abilities required by a system of course performances is inferred 
by finding the minimum number of postulated abilities which yield a good 
approximation of the grades. For example, if two abilities had been 
post^alated in the preceding example, there would be^only two a 's' and two 
z 's, and the residuals would be different than when the three ability 
system is used to approximate the grades. The sum of squares of these 
residuals for the two-ability system can be compared with the sum of 
squares of residuals for the three-ability system to help decide if the 
addition of a third ability provided a better fit to the data. Of course, 
the three-ability system uses more parameters than the two-ability system; 
it uses as rany more z 's as there are students and as many more a 's as 
there are courses. Use of so many more parameters ensures that the three- 
ability system will fit the data better than will the two-ability system 
but trivial reductions in the sum of squares of residuals will not be con- 
sidered evidence which confirms the necessity to accept the third factor. 

In summary of the methodological points above, there are four kinds 
of quantities that result from the factor analysis. One is a set of 
parameters, b 's in the discussion above, which adjust for the difficulty 
of the courses. Another set of parameters, z 's in the discussion above, 
represent the student's scores on the inferred variables which underlie the 



grades. These z *s combine Into weighted sums to approximate the grades, 
and the weights in these sums are the a *s of the discussion above; the 
larger a weight the greater the dspendence of the grades on that ability. 
Finally, the goodness of fit of a particular number of factors is indicated 
by the sum of squares of residuals, a smaller^ value indicating a better fit. 

Data 

Data which were obtained from an American law school consisted of 
transcripts of law school performance for 116 students. Along with a 
student's transcript was included his LSAT score as well as his Writing 
Ability and Background scores. These data were sorted by course and 62 
courses were identified for which grades for seven or more students were 
available. In addition, information was provided as to which instructors 
taught the various' courses. In all, 3315 grades were the main subject of 
the analysis. 

Analysis of Factors by Course 

The first part of the analysis- was intended to determine the number of 
factors. Table 1 presents the sums of squares of residuals for various 
numbers of factors and a correlation interpretation of these data. The 
bench-mark value taken for this analysis is the fit of a single mean value 
for all of the observations. The sum of squares of differences between 
that value and the observed grades is given in the^^first line of Table 1. 
Line two of the table gives the sum of squares of residuals when the 
-average grade for a course is used to approximate the grades in the course; 
62 such averages were calculated, there being 62 courses, and it can be noted 
that the sum of squares of residuals in line two is not very different than 



that in line one* The difference between the rt«ro sums of squares is given 
in the third column of Table 1, and that difference is interpreted as a 
multiple correlation in the fourth column of that table. The multiple 
correlation interpretation is developed just as in other research which 
uses correlation ideas, where the squared multiple correlation is equal 
to the percent of sums of squares accounted for by the predictors and the 
sums of squares are taken around a single criterion average. As far as 
predicting the grades is concerned, one could replace a predictor which 
correlated .36 with the grades with the^ coarse average and predict just as 
well* Note that fitting one factor improves the fit markedly, increasing 
the correlation by 30 points, but after that the iinprojiteroent in fit is not 
large. Usually, when one is selecting predictprs in a regrcTssion situation 
one requires that the addition of a test to a prediction set adds at least 
.005 to the multiple correlation coefficient for the selection of variables 
to continue. This increase comes about because of the regression weight 
for the added variable, but in the present situation one adds not onl. 
regression weight for each course but also a factor score for each student. 
Each factor requires the addition of 150 or more parameters yet the gain 
noted after one factor in Table 1 is .1 or less. 

Insert Table 1 about here 

A plot of the two-factor solution a 's was developed and presented 
in Figure 1 to suggest substantive hypotheses even though the analysis of 
residuals does not establish the need for acceptance of the second factor. 



Insert Figure 1 about here 



Virtually all the points lie in the shaded area on the plot with the 
exception of the extreme deviants whose course titles are given along 
with the number of students enrolled and the time the course is offered. 
Note tnat those courses which appear deviant are not entirely from a single 
year, nor do they seem restricted to courses which reflect a conoon content 
theme. 

In tho previous paragraph the plot of the a 's for the two-factor 
solution was examined merely for the sake of finding interesting possible 
interpretations of the data. The case for examining the three- factor 
solutions is somewhat stronger, as can be seen by examining the column 
labeled "Sums of Squares Attributable to Parameters" in Table 1. Notice 
that in this column the gain by fitting the first factor is quite large 
(247.78 - 74.52 - 173.26), the gains by fitting the second factor (319.74- 
247.78 - 71. ?6) and the third (391.54 - 319.74 - 71.80) are about the same, 
and the gain by fitting the fourth factor is smaller (413.24 - 391.54 « 21.8). 
Some might take the equality of the reductions by the second and third 
factors, followed by the rather small reduction supplied by tl^e fourth as 
evidence in favor of accepting the third factor. For this reason. Figure 2 
was developed for interpretative purposes. Figure 2 is a plot of modified 
a •s obtained with the three-factor solution. The modification used is 
consistent with the method of extended vectors (Thurstone, 1947) which 
allows the examination of a three-dimensional configuration using a two^ 
dimensional space (piece of paper). In this application of the method of 
extended vectors the three-dimensional plot of a 's was projected onto a 
plane defined by a coordinate of unity for the axis corresponding with the 
third factor. If the a 's describe a single factor except for a few 
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outliers^ the plot of Figure 2 would appear essentially as a streak of 
points on a lino with a few points off the line. It does look a little 
that wayy but three axes have been drawn in to indicate what one night 
wish to interpret as three streaks* though the evidence for these three 
' streaks seems not very.strcng. The points defining the extremes of the 
streaks have been-nu8d)ered^ along with some outliers so that these partic* 
ular points and streaks can be discussed and presented in Tibbie 1. The 
courses are arranged by the vector with which they are associated* if they 
are not simply a deviant* and one may note that Course 950-11 is included 
twice as it gives ah end point of both vectors A and B. The footnoted 
entries in Table 2 are ones which occur because the numerical operations 
involved in the extended vector computations require a division by « number 
which was very small in the case of these courses, the footnotes dictate 
caution in placing confidence on interpretations based on the footnoted 
loadings. Table 2 also contains the year in which the courses are 
normally taken. 

Insert Figure 2 and Table 2 about here 

The author's interpretation of Figure 2 and Table 2 is that the separa- 
tion of vectors A and B is somewhat forced* but that perhaps vector C 
indicates some trend. If so* that trend seems not to be one of differentiat- 
ing socially oriented* third-year courses from the more traditional first- 
year courses. In fact* if the deviants of Table 2 are attributed to sampling 
instability due to division by a small number and if vectors A and B are 
considered to be essentially the same* then the space would be considered 
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«i two-di«ensional space. But the two-fector solution has been examined 
and not found to be enlightening. The evidence for a second or third 
factor still does not sees convincing though Interpretations of this kind 
are subjective and the reader Is not bound by technical considerations to 
agree. 

Table 3 Is a table of intercor relet Ions of LSAT, Writing Ability, 
Background. Fll (the factor scores fro« the single factar analysis), and 
F12 and F22 (the first and second factors fro« the two-factor analysis). 
The validity of the three : , ts for predicting Fll was .39 vith standard 
score^regressloh weights of .36. .14, and -.13 for LSAT. Writing AblUty* 
and Background, respectively. To exaidhe the coi'reiation of the predlctori 
with the two-factor systeit the F12 and F22 scores were combined Into a 
coiaposlte that was uncorrelated with FU. This cos|)oslte would represent 
that part of the factor variance that Is unrelated to the first facto'^-- 
essentlally nw variance Introduced by the inclusion of the second factoir* 
One would be quite Interested to observe a substantial relationship betweieiS 
this compoFlte and the predictors because it would indicate the existence 
of systematic variance in the two-factor system and a need for different 
combinations of the predictors for the two factors. However, the multiple 
correlation between this composite and the predictors was only .03. Fro«^ 
the standpoint of prediction, using the LSAT, Writing Ability, and Background, 
there is mainly one component of the two-factor system and that component 
is the part that is predictable using the tests. The multiple correUtlon 
between the two factors and Fll is .96. 

Insert Table 3 about here 
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Analysis of Residuals,for_ Instructor Effects 

One problem in analyzing for instructor effects is that it is quite 
difficult to separate instructor effects from course effects.. For example, 
if an iiastructor taught only one course and that course was the only one 
he taught, tfren there would be absolutely no way to separate the course 
effects from the instructor effects. What is heeded for such ah analysis 
is a situation where an. instructor teaches a number of courses and the 
courses are taught by a number o£ instructors. The data for such an 
analysis wpuld .requ^^^^ carry not only the course 

;t:he student took, but som^^ indication^ such as an instructor code or a 
section code, of whom the instructor was. In the present data this con- 
dition holds, but only partiaily. It holds in that the sections of a 
course can be distinguished if they are offered in different terms, but 
not if offered in the same term, unless the sections in the same term 
are taught by the same instructor. The couf selCaeiirif ication that appears 
on the transcript indicates the course and grade but not the section 
number . 

iln an attempt to circumvent these difficulties it was decided to 
approach the problem section by section, this was possible because 
sections of a course were treated as equivalent for purposes of the factor 
analysis; and if more than one instructor gave a course, constant differ- 
ences in their grading practices could be studied by examining the residuals 
for students in each instructor's section. Therefore, a study of the 
residuals by section would, when correlated with the identity of the 
instructors teaching the section, identify deviant grading practices. The 
examination of these residuals was restricted to significance testing of the 



average residual by section; the variance used for this testing was the 
sum of squares of residuals for the one factor system divided by 3315, the 
number of observations. The tests were done as if the variances were known 
and the t -tables were used, taking the degrees of freedom as equal to the 
nul>.er of observations. The formula for t^ was that for a test of signif- 
icance from zero where the standard deviation of the observations is known. 
It should be pointed out that these tests are probably rather low in power 
isihce the fitting of the factor roodel forces I ha residuals to average at 
ferd for each course ^ thus^ if a result is indicated as sighif icaht by 
the test used, it is probably a real one. 

Insert Table 4 about here 

In all, 153 t^ -^tests were made with 12 being significant at the 2% 
level or more. Table 4 presents the comparison of. the number of tests 
found significant for various confidence levels and the number of signif- 
icant tests expected on a chance level. It can be noted that at the 2% 
level or less there are more significant results than might be expected 
by chance. Table 5 presents the course name, number, the instructor 
number (s), the significance level, and the average residual expressed 
oh the grade-point scale, both as a four-point scale and on the grade- 
point scale at this law school, which ranged from 0 to 100. Note that 
in table 5, Instructor 4 and Instructor 7 appear twice as the only 
instructors of a section. Instructor 4 appears to be an easy grader, and 

Insert Table 5 about here 
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Ihs true tor 7 appears to be a hard grader. Table 6 contains the course 
title y number of cases » and average residuals on the two scales for 
Instructor 4 and also for Instructor 7. The average residual for Instructor 
4' is -8*7 on the law school* s scale indicating that he tended to give grades 
alxDost 9 points below that predicted by the one-factor model. Instructor 7 
gave grades 1.2 points above those predicted by the one-factor system. 
Notfe that his average residuals are mixed in sign and that his bias of 

1.2; points is hot really very large when compared with the lOO-poiht scale. 

- - - . - * 

Insert Table 6 about here 

Summary 

Grades of 116 law school students were obtained from transcripts, 
along with LSAT» Writing Ability, and Background scores^ The grades were 
factored, and an analysis of the goodness of fit of one- through four-factor 
systems indicated that the system of grades was essentially one-factor in 
nature. A plot of factor loadings for the various courses was made for the 
two^ and three-factor systems, and it was found that there were a few 
courses somewhat different than the rest but their nature was not clearly 
identifiable. The two-factor system was decomposed into two uncorrelated 
components, one which correlated .96 with the single factor scores, and the 
qther which correlated zero with the single factor scores. This latter 
component had a multiple correlation of .03 with LSAT, Writing Ability, and 
Backgrottnd scores* These analyses indicated that no persuasive evidence 
of more than one factor could be adduced. 



The factor system did not, of course, provide an exact fit to the 
grades, and the residuals produced were used to study the possible exist- 
ence of instructor bias. Two instructors were found whose residuals were 
on two occasions significantly different from zero, and the average 
residuals were presented for all of the sections for which these men could 
be identified as the instructors. One was found to be a slightly easy 
;grader, arid the other was fouitd to.be a hard grader. 

The results are esseritially in agreement: with another study in the 
graduate business school context (Boldt , 1976) , where data from two schools 
were examined and a sirigle factor was found at each location. 
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; ' Table 1 

M , Analysis of Sums of Squares of Residtaals 



Parameter Fit 


Sums of Squares 
of Residuals 


Sums of Squares 
Attributable to 
Parameters 




Grand Mean 


577.36 






b's 


502.84 


74.52 


-.36 


b's and 1 Factor 


. 329.58 


247.78 


.66 


b's and 2 Factors 


257.62 


319.74 


.74 - 


b's and 3 Factors 


185.82 


391.54 


.82 


b's and 4 Factors 


i64.02 


413.34' 


.85 



I ^Multiple correlation 



t 



I 



Table 2 

Course Numbers, Titles* Number of Students Taking, and 
the Year Taken for Courses Ranked on Figure 2 



Nu]d>er 





of 

Students 


Course 
Number 


Title 


Year 
Taken 


Vector A 












llA 


815* 


Practice arid Procedure I 


1 




107 


852* 


Biisiriess Associations II 


2 


-- 


85 - 


- 828 


Workmen's Comperisatiori 


2 or 3 




19 * 


950-11 


Iriternational Law 


3 


Vector B 












111 


860* 


taxatibri I 


2 




107 


817 


Practice arid Procedure II 


2 




19 


950-11 


Iriternational Law 


3 


Vectpr G 












25 


930 


Gov't Regulatiori 6£ Business 


3 




98 


824 


Legal Research Tutorial 


1 




11 


872 


Land Use Planning 


3 




101 


8-55 


Corporation Finance Law 


3 


Deviants 












13 


950-25* 


Fiduciary Administration 


3 




7 


825* 


Legislation 


2 or 3 




52 


885* 


Creditors' Rights 


2 




111 


876* 


Trusts arid Estates 


1 



Unstable due to small loadings on Factor 3. 



Table 3" 

Correlations of Predictor and Factor Scores 





LSAT 


Writing 


Background 


Fll^ 


F12^ 


.F22^ 


LSAT 


i.ob 


.56 


.54 


.36 


.10 


, -.33 


Writing 




1.00 


.54 


.27 


.08 


-.26 


Background 






1.00 


.14 


.02 


-.10 


Fll 








1.00 


.35 


-.91 


F12 










1.00 


-.05 


F22 












1.00 



^Fil, F12, and F22 are defined in the text. 
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Table A 

Table of Confidence Leyels of t^Tests 





p > .2 


.2 > p > .1 .1 > p > .05 


.05 > P > .02 


.02 > p 


Observed 


129.0 


7.0* 3.0 


2.0 


12.0 


Expected 


122. A 


15.3 7.7 


A. 6 


3.0 



er of significance testr. 



-19- 



CO 

•O 
•H 

10 

& 

00 
CO 

I 



c 

CO 

a 



CO 

M 
u 

I 

a 

CO 

§ 

CO 



CO 

•o 

(0 

PC 

0) 
00 
CO 

u 



o 
o 
4: 
o 

CO 
1-4 



e 



O CO 



0) 



01 

o 
e 

CO 

o 

MH > 
•H 01 

CO 



M 
O 

u 

O 0) 

SI 

e 



9 
o 



0 


• 


10 
• 


• • 


• • 


CM CM 
• • • 


CO CM 




CO 


so 0\ 




^ ^ ^ 


1 




1 


1 


H 


fH fH fH 
1 1 


00 


0 




00 


*^ CM 


OS 


CO 


CO 




CM fs. 


*^ CM 


10 m 








1 




< 1 


f 












- 

CO 00 








0 


ps sO OS 






CO 


H 




















fH 


CM 




iH CM 


0 r-4 0 


0 0 


0 


0 


0 0 


0 0 


000 












V 






CM 


























sO P 




«^ sO «9 












CO 


m so 




00 




* 
















0) 




































tJ 












0)S 










g 


u 


• 










0 




Si 






0) : : 


u 


6 


CO 












f-3 


















M 


id 




fH 


U 










CO 


U 






0) 




•H 


a z 






u 




0 


0 


< 


< 








0 










» 








U 






u 


CO 


CO 


m 






0 


oos 


oos z 






0 




0) 


0) 








:g = 










CM 




M CM 


CO CO CO 


in in 


00 


1 










CM 


00 


SO SO 


SO so 


so so so 


0000 


00 


00 


OS OS 


OS OS 


OS OS OS 



SO 



CM 
CM 



CM 
O 



o 

U 

CO 
0) 
0) 
0) 



so 
I 



SO 
OS 



Table 6 

Course Data for Instructors A and 7 



'^^^ Average Residual 



Number A*-Polnt Law School *s 
Title . Cases Scale Scale 



Course 

- Instructor A 

Legal Aid Intern • 
II II II 

II II 11 

fax Planning 
legal Aid 

Average 

Instructor 7 

federal Courts & Federal Systems 

Relationar Torts 
II II 

If II 

Legal Aid 
II II 

II II 

Law Journal Itote Editing 
If If .11 11 

Practice Court 
Average 



9 -.59 -1A.7 

9 ^.57 -1A.2 

7 -.03 -.6 

1 -.29 -7.2 

3 .28 7.1 

-.35 -8.7 



35 0 0 

16 .05 1.3 

3 -.03 -.7 

39 -.03 -.8 

10 ' .AA 11.1 

lA .22 5.6 

7 -.29 -7.2 

6 .08 1.9 

1 -.07 -1.8 

1 .26 6.6 
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